---
title: vertex-ai
---

The `vertex-ai` provider is used to interact with the Google Vertex AI services.

<Tip>As of BAML 0.85.0, `vertex-ai` now supports Anthropic models!</Tip>


Example using a Vertex API Key (Express Mode):
```baml BAML
client<llm> MyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    location us-central1 // or "global"
    project_id my-project-id
    query_params {
      key env.VERTEX_API_KEY
    }
  }
}
```

## Authentication

### Using a Vertex API Key (Express Mode)

<Info>
  To get started quickly, we recommend using Express Mode with a Vertex API Key.
  This avoids service account setup and works well for prototyping.

  See Google's guide: [Use Vertex API keys (Express Mode)](https://cloud.google.com/vertex-ai/generative-ai/docs/start/api-keys?usertype=expressmode).

  See also [Express mode overview](https://cloud.google.com/vertex-ai/generative-ai/docs/start/express-mode/overview).
</Info>

When using a Vertex API Key, set the `key` query parameter and specify your `project_id` and `location`:

```baml BAML
client<llm> VertexApiKeyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    location us-central1 // you can also use "global"
    project_id my-project-id
    query_params {
      key env.VERTEX_API_KEY
    }
  }
}
```

**When in doubt, check the 'cURL' tab in the playground to see the exact request being sent!**

Notes:
- `project_id` cannot be inferred when using an API key; set it explicitly.
- Keep `credentials` unset when using an API key, so BAML does not prefer service account auth.

#### Using a Vertex API Key in the playground
You should see the `VERTEX_API_KEY` environment variable in the playground API Keys dialog. You can set it there and you're all set!


### Using Google Application Credentials

If no vertex api key is set, BAML will by default try to authenticate using [application default
credentials](https://cloud.google.com/docs/authentication/application-default-credentials)


```
client<llm> MyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    location us-central1
    project_id my-project-id
    // we will by default try to use this form of authentication.
    credentials env.MY_APPLICATION_CREDENTIALS_CONTENT
  }
}
```
This is what the MY_APPLICATION_CREDENTIALS_CONTENT environment variable looks like:
```json
MY_APPLICATION_CREDENTIALS_CONTENT={
  "type": "service_account",
  "project_id": "my-project-id",
  "private_key_id": "string",
  "private_key": "-----BEGIN PRIVATE KEY-----string\n-----END PRIVATE KEY-----\n"
  ...other fields...
}
```
BAML accepts this blob as a string, a path to a file, or a JSON object.


#### More details on Google Application Credentials
Here is the order of authentication:
- If `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set, it will use the specified service account
- If you have run `gcloud auth application-default login`, it will find the
  credentials generated by `gcloud` by the path convention. Note that you will still
  need to set either `options.project_id` or the `GOOGLE_CLOUD_PROJECT` environment variable.
- If running in GCP, it will query the metadata server to use the attached service account
- If `gcloud` is available on the `PATH`, it will use `gcloud auth print-access-token`

### Requirements

You need to use an account with a ProjectID that has been authorized to use Vertex.
When administering your Google Cloud account, be sure to enable Vertex, and set up ADC:

```bash
gcloud auth application-default login
```

If you're using Google Cloud [application default
credentials](https://cloud.google.com/docs/authentication/application-default-credentials), you
can expect authentication to work out of the box.

Setting [`options.credentials`](#credentials) will take precedence and force `vertex-ai` to load
service account credentials from that file path.

### Playground
To use a `vertex-ai` client in the playground, you need to run `gcloud
auth application-default login` in the terminal and set the
`GOOGLE_CLOUD_PROJECT` environment variable in the "API Keys" dialog. The
playground will then use these credentials to auth all Vertex API calls.

## Debugging

<Accordion title='Authentication'>

If you're having issues with `vertex-ai` authentication, you can try setting
`BAML_INTERNAL_LOG=debug` to see more detailed logs.

To understand these logs, it'll help to understand the auth implementation of the `vertex-ai` provider.

The `vertex-ai` provider uses one of 3 strategies to authenticate with Google Cloud:

- `AuthStrategy::JsonString(value: String)` - parse `value` as a JSON
  object, and use that to resolve a service account
- `AuthStrategy::JsonFile(path: String)` - read the file at `path` (relative to
  the process' current working directory), parse it as a JSON object, and use that
  to resolve a service account
- `AuthStrategy::SystemDefault` - try 3 strategies in order:
    - resolve credentials from `.config/gcloud/application_default_credentials.json`; else
    - use the service account from the GCP compute environment by querying the metadata server; else
    - check if `gcloud` is available on the `PATH` and if so, use `gcloud auth print-access-token`

We choose one of the three strategies based on the following rules, in order:

1. Is `credentials` provided?
     - If so, and it's a string containing a JSON object, we use `AuthStrategy::JsonString` with `credentials`.
     - If so, and it's a JSON object, we use `AuthStrategy::JsonObject` with `credentials` (this is probably only
       relevant if you're using the [`ClientRegistry`](/ref/baml_client/client-registry) API in `baml_client`).
     - If so, but it's just a regular string, use `AuthStrategy::JsonFile` with `credentials`.
1. Is `GOOGLE_APPLICATION_CREDENTIALS` set?
     - If so, and it looks like a JSON object, we use `AuthStrategy::JsonString` with `GOOGLE_APPLICATION_CREDENTIALS`
     - If so, but it's just a regular string, use `AuthStrategy::JsonFile` with `GOOGLE_APPLICATION_CREDENTIALS`
1. Else, we use `AuthStrategy::SystemDefault`

</Accordion>

<Accordion title='Request protocol'>

We use the REST API to send requests to Vertex AI, and you can debug these using
the BAML playground and switch from showing "Prompt Preview" to "Raw cURL", which
will show you the exact request the BAML runtime will construct and send.

Non-streaming requests will use `{base_url}:generateContent`:
```
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:generateContent
```

Streaming requests will use `{base_url}:streamGenerateContent?alt=sse`:
```
https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/google/models/${MODEL_ID}:streamGenerateContent
```

</Accordion>

## BAML-specific request `options`
These unique parameters (aka `options`) modify the API request sent to the provider.

You can use this to modify the `headers` and `base_url` for example.

<ParamField path="base_url" type="string">
  The base URL for the API.

  **Default**: inferred from the `project_id` and `location` using the following format:

  ```
  https://{LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/{LOCATION}/publishers/google/models/
  ```

  If the location is `global`, the base URL will be:
  ```
  https://aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/global/publishers/google/models/
  ```

  Can be used in lieu of the **`project_id`** and **`location`** fields, to manually set the request URL.
</ParamField>


<ParamField
  path="project_id"
  type="string"
>
  The Google Cloud project ID hosting the Vertex AI service you want to call.

  **Default**: inferred from the provided credentials (see [`Authentication`](#authentication)).
</ParamField>

{/*The anchor is placed above "location" and not "credentials" because this will ensure that "credentials" is
visible on-screen when the user navigates to #credentials, due to how Fern renders its HTML layout.*/}

<a name="credentials"></a>

<ParamField
  path="location"
  type="string"
  required
>
  Vertex requires you to specify the location you want to serve your models
  from. Some models may only be available in certain locations.

  Common locations include:
  - `us-central1`
  - `us-west1`
  - `us-east1`
  - `us-south1`

  See the [Vertex AI docs](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations#united-states)
  for all locations and supported models.

</ParamField>

{/*The anchor is placed above "credentials" and not "credentials_content" because this will ensure that "credentials_content" is
visible on-screen when the user navigates to #credentials_content, due to how Fern renders its HTML layout.*/}

<a name="credentials_content"></a>

<ParamField
  path="credentials"
  type="string | object"
>

  This field supports any of 3 formats:

    - A string containing service account credentials in JSON format.
    - Path to a file containing service account credentials in JSON format.
    - A JSON object containing service account credentials.

  See [Authentication](#authentication) and [Debugging](#debugging) for more information.

  **Default: `env.GOOGLE_APPLICATION_CREDENTIALS`**

  <Accordion title='Example: string'>
    ```baml BAML
    client<llm> Vertex {
      provider vertex-ai
      options {
        model gemini-2.5-pro
        location us-central1
        // credentials can be a block string containing service account credentials in JSON format
        credentials #"
          {
            "type": "service_account",
            "project_id": "my-project-id",
            "private_key_id": "string",
            "private_key": "-----BEGIN PRIVATE KEY-----string\n-----END PRIVATE KEY-----\n",
            "client_email": "john_doe@gmail.com",
            "client_id": "123456",
            "auth_uri": "https://accounts.google.com/o/oauth2/auth",
            "token_uri": "https://oauth2.googleapis.com/token",
            "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
            "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/...",
            "universe_domain": "googleapis.com"
          }
        "#
      }
    }

    ```
  </Accordion>

  <Accordion title='Example: file path'>
  In this case, the path is resolved relative to the CWD of your process.

  ```baml BAML
  client<llm> Vertex {
    provider vertex-ai
    options {
      model gemini-2.5-pro
      location us-central1
      credentials "path/to/credentials.json"
    }
  }
  ```
  </Accordion>

  <Accordion title='Example: JSON object'>
    ```baml BAML
    client<llm> Vertex {
      provider vertex-ai
      options {
        model gemini-2.5-pro
        location us-central1
        // credentials can be a block string containing service account credentials in JSON format
        credentials {
          type "service_account",
          project_id "my-project-id",
          private_key_id "string",
          private_key "-----BEGIN PRIVATE KEY-----string\n-----END PRIVATE KEY-----\n",
          client_email "john_doe@gmail.com",
          client_id "123456",
          auth_uri "https://accounts.google.com/o/oauth2/auth",
          token_uri "https://oauth2.googleapis.com/token",
          auth_provider_x509_cert_url "https://www.googleapis.com/oauth2/v1/certs",
          client_x509_cert_url "https://www.googleapis.com/robot/v1/metadata/...",
          universe_domain "googleapis.com"
        }
      }
    }
    ```
  </Accordion>

</ParamField>

<ParamField
  path="credentials_content"
  type="string"
>
  <Warning>
    Since the BAML playground now allows using `gcloud auth application-default login`, to
    authenticate wih GCP, we will soon be deprecating `credentials_content`.
  </Warning>

  A string containing service account credentials in JSON format.

  See [Authentication](#authentication) and [Debugging](#debugging) for more information.

  **Default: `env.GOOGLE_APPLICATION_CREDENTIALS_CONTENT`**

  <Accordion title='Example'>
    ```baml BAML
    client<llm> Vertex {
      provider vertex-ai
      options {
        model gemini-2.5-pro
        location us-central1
        // credentials_content is a block string containing service account credentials in JSON format
        credentials_content #"
          {
            "type": "service_account",
            "project_id": "my-project-id",
            "private_key_id": "string",
            "private_key": "-----BEGIN PRIVATE KEY-----string\n-----END PRIVATE KEY-----\n",
            "client_email": "john_doe@gmail.com",
            "client_id": "123456",
            "auth_uri": "https://accounts.google.com/o/oauth2/auth",
            "token_uri": "https://oauth2.googleapis.com/token",
            "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
            "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/...",
            "universe_domain": "googleapis.com"
          }
        "#
      }
    }

    ```
  </Accordion>
</ParamField>


<ParamField
  path="model"
  type="string"
  required
>
  The Google model to use for the request.


| Model | Input(s) | Optimized for |
| --- | ---  | --- |
| `gemini-2.5-pro`  | Audio, images, videos, and text | Complex reasoning tasks such as code and text generation, text editing, problem solving, data extraction and generation |
| `gemini-2.5-flash`  | Audio, images, videos, and text | Fast and versatile performance across a diverse variety of tasks |
| `gemini-1.0-pro` | Text | Natural language tasks, multi-turn text and code chat, and code generation |

See the [Google Model Docs](https://ai.google.dev/gemini-api/docs/models/gemini) for the latest models.
</ParamField>

<ParamField path="headers" type="object">
  Additional headers to send with the request.

Example:
```baml BAML
client<llm> MyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    project_id my-project-id
    location us-central1
    // Additional headers
    headers {
      "X-My-Header" "my-value"
    }
  }
}
```
</ParamField>

<ParamField path="query_params" type="object">
  Query string parameters appended to the request URL.

Example (use a Vertex API Key with Express Mode):
```baml BAML
client<llm> MyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    project_id my-project-id
    location us-central1
    query_params {
      key env.VERTEX_API_KEY
    }
  }
}
```

  When using an API key, omit `credentials` and set `project_id` explicitly.
</ParamField>

<Markdown src="/snippets/role-selection.mdx" />

<Markdown src="/snippets/allowed-role-metadata-basic.mdx" />

<Markdown src="/snippets/supports-streaming.mdx" />

<Markdown src="/snippets/finish-reason.mdx" />

<Markdown src="/snippets/media-url-handler.mdx" />

<Note>
  Vertex AI uses `send_url_add_mime_type` by default for images and audio, which ensures MIME type information is included. This may require downloading the content to detect the MIME type if not provided.
</Note>

## Provider request parameters
These are other parameters that are passed through to the provider, without modification by BAML. For example if the request has a `temperature` field, you can define it in the client here so every call has that set.

Consult the specific provider's documentation for more information.
<ParamField
  path="safetySettings"
  type="object"
>
  Safety settings to apply to the request. You can stack different safety settings with a new `safetySettings` header for each one. See the [Google Vertex API Request Docs](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference) for more information on what safety settings can be set.

```baml BAML
client<llm> MyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    project_id my-project-id
    location us-central1

    safetySettings {
      category HARM_CATEGORY_HATE_SPEECH
      threshold BLOCK_LOW_AND_ABOVE
      method SEVERITY
    }
  }
}
```
</ParamField>

<ParamField
  path="generationConfig"
  type="object"
>
  Generation configurations to apply to the request. See the [Google Vertex API Request Docs](https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference) for more information on what properties can be set.
```baml BAML
client<llm> MyClient {
  provider vertex-ai
  options {
    model gemini-2.5-pro
    project_id my-project-id
    location us-central1

    generationConfig {
      maxOutputTokens 100
      temperature 1
    }
  }
}
```

</ParamField>

For all other options, see the [official Vertex AI documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal).

## Publishers Other Than Google

If you are using models from publishers other than Google, such as Llama from
Meta, use your project endpoint as the `base_url` in BAML:

```baml
client<llm> VertexLlama {
  provider vertex-ai
  options {
    base_url "https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/endpoints/"
    location us-central1
  }
}
```

For anthropic  

```baml
client<llm> VertexClaudeSonnet {
  provider vertex-ai
  options {
    model "claude-sonnet-4"
    anthropic_version "${ANTHROPIC_VERSION}"
    base_url "https://${LOCATION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${LOCATION}/publishers/anthropic/models"
  }
}
```
